Uncertainty,
standard errors and
confidence intervals
LAST WEEK: Jennifer undertook a determined journey to find out just how clever Jessie (left) is compared to all the other dogs in the population.
THIS WEEK: What if we don’t know anything about the population?
The whole game
- We want to estimate some population value (e.g. some average value - a mean)
- Confidence intervals help us quantify uncertainty around that estimate
- To construct a confidence interval, we use the standard error
- We can estimate the SE by dividing the standard deviation by the number of people in our sample (specifically, by its square root):
\[ SE = \frac{\sigma}{\sqrt N} \]
- Lower and upper limits of the confidence interval can be estimated as:
\[ \text{CI limits} = mean \pm (1.96 \times{SE}) \\ \]
Doomscrolling
“… refers to a unique media habit where social media users persistently attend to negative information in their newsfeeds about crises, disasters, and tragedies.”
- Sharma, Lee, and Johnson (2022)
Research question...
How much does an average person doomscroll?
Uncertainty in research and estimation
The problem: Each time we take a sample, we get a different estimate.
How do we know if our estimate is accurate and close to the real population value?
Uncertainty in research and estimation
The problem: Each time we take a sample, we get a different estimate.
How do we know if our estimate is accurate and close to the real population value?
Sampling distribution revisited
- We can plot the estimates in a histogram to see how they’re distributed
- x axis shows the spread of the values, y axis shows how many times each value occurs
Sampling distribution revisited
- We can repeat the process infinite number of times - as long as our sample is large enough, the sampling distribution of the estimates will be normal
- What is “large enough”? Some textbooks say 30, but more is often needed in real research.
Normal distribution is awesome
- Normal distribution is useful, because we know a lot about it - a good way to describe it is by using a mean (central value) and a standard deviation (spread of the scores)
- Standard error is the term we use for standard deviation when talking about sampling distributions
Normal distribution is awesome
- We know that 95% of scores will fall within approximately 2 standard errors from the mean (or 1.96 if we want to precise)
- We can use this knowledge to construct an interval around the mean showing us a range of plausible values for the population value
Normal distribution is awesome
- We know that 95% of scores will fall within approximately 2 standard errors from the mean (or 1.96 if we want to precise)
- We can use this knowledge to construct an interval around the mean showing us a range of plausible values
Confidence intervals
Average doomscrolling time for the sample: 101 minutes
Standard deviation: 12.19
Standard error: 6.095
\[ \text{Lower CI limit} = \text{sample mean} - 1.96 \times\text{SE} \\ \text{Upper CI limit} = \text{sample mean} + 1.96 \times\text{SE} \]
\[ \text{Lower CI limit} = 101 - 1.96 \times6.095 = 89.054\\ \text{Upper CI limit} = 101 + 1.96 \times6.095 = 112.946 \]
Confidence intervals for small samples
Remember: sampling distribution of the mean will have a normal shape as long as the sample size large enough
Smaller samples don’t approximate the normal sampling distribution very well. Because of this, we can’t rely on the value 1.96 to give us accurate intervals.
Instead, we can use the t-distribution
The t-distribution
The “critical t value” - i.e. the value of t which will give us the most accurate estimate of the confidence interval - depends on degrees of freedom (df), which in our case are related to the sample size.
When working with the mean, the degrees of freedom are calculated as N - 1.
The t-distribution
Instead of multiplying the standard error by 1.96, we multiply by the critical t value.
For example, in our sample of 4, the df is 4 - 1 = 3. Move the slider to df = 3 to see that the critical t value for 3 is 3.182
t-based confidence intervals:
Average doomscrolling time for the sample: 101 minutes
Standard error: 6.095
Critical t value: 3.182
\[ \text{CI Limits} = \text{mean} \pm3.182 \times\text{SE} \\ \text{CI Limits} = 101 \pm3.182 \times\text{6.095} \\ \text{CI Limits} = [81.606, 120.394] \]
- Compare the new confidence interval [81.61, 120.394] with the confidence interval we got using the value 1.96: [89.05, 112.95]. The new CI is wider!
t-based confidence intervals:
This is to be expected - we have a tiny sample (N = 4), so there is a lot of uncertainty around whether the estimate of 101 minutes is actually representative of the population.
The larger the sample, the tighter the confidence intervals,. because the critical t gets smaller and smaller (note how t approaches 1.96 as the sample size (df) increases)
We take samples over and over again, compute the mean, and construct confidence intervals around that mean - 95% of them will contain the population value, the remaining 5% will not.
This is known as an interval with 95% coverage. 95% is the most common value that we choose, but it can take on other values as well (e.g 50%, 90%, 99%).
How *not* to interpret confidence intervals
Nope:
“We can be 95% confident that the population value falls between 81.61 and 120.39.”
How *not* to interpret confidence intervals
Also nope:
“There is 95% probability that the population value falls between 81.61 and 120.39.”
How to interpret confidence intervals
Correct interpretation
ASSUMING THAT our sample is one of the 95% producing confidence intervals that contain the population value, then the population value for time spent doomscrolling per day falls somewhere between 81.61 and 120.39 minutes.
The whole game (again)
- We want to estimate some population value (e.g. some average value - a mean)
- Confidence intervals help us quantify uncertainty around that estimate
- To construct a confidence interval, we use the standard error which we can estimate as:
\[ SE = \frac{\sigma}{\sqrt N} \]
- Lower and upper limits of the confidence interval can be estimated as (replacing 1.96 with critical t for small samples):
\[ \text{CI limits} = mean \pm (1.96 \times{SE}) \\ \]
The bigger picture…
When interpreting estimates and confidence intervals for your sample - always consider them as just one of many different possible estimates
This is why replication is important in science - our sample could easily be the one that misses the population value
Always be vary of studies placing too much confidence on a single finding
